RESEARCH  BRIEF 


RAND 

CORPORATION 


Evaluating  Innovative  Leader  Development  in  the 
U.S.  Army 


The  U.S.  Army’s  shift  from  a  doctrine  of  “command 

and  control”  to  “mission  command”  calls  for  adaptable 
soldiers  and  leaders — individuals  who  can  rapidly  recog¬ 
nize  changes  in  the  environment,  identify  critical  elements  in 
unfamiliar  situations  with  less-than-perfect  information,  and 
facilitate  timely  action  to  meet  new  requirements,  all  while 
under  considerable  stress.  The  principles  of  mission  com¬ 
mand  also  emphasize  leaders  who  value  a  bottom-up  rather 
than  a  top-down  approach  and  who  develop  teams  that  can 
anticipate  and  manage  transition. 

In  this  context,  the  Army’s  Asymmetric  Warfare  Group 
(AWG)  designed  and  implemented  the  Asymmetric  Warfare 
Adaptive  Leader  Program  (AWALP) — a  ten-day  course  to 
enhance  adaptability  in  leaders  and  promote  innovative  solu¬ 
tions  in  training  for  and  conducting  unified  land  operations. 
The  course  design  is  based  on  a  theory  of  adaptive  perfor¬ 
mance  that  posits  eight  dimensions  of  such  performance, 
such  as  solving  problems  creatively,  dealing  with  changing 
or  ambiguous  situations,  interpersonal  adaptability,  cultural 
adaptability,  and  making  decisions  under  stress.  Although 
much  Army  training  focuses  on  standardized  procedures  for 
accomplishing  tasks,  AWALP  places  more  emphasis  on  an 
outcomes  approach,  focusing  on  the  results  the  commander 
intends  to  achieve.  This  approach  encourages  trainees  to 
take  the  initiative  to  adjust  actions  to  adapt  to  the  situation, 
which,  in  turn,  requires  independent  thinking  and  problem 
solving.  Thus,  AWALP  exemplifies  mission  command  prin¬ 
ciples  for  both  course  content  and  how  the  course  is  taught. 

RAND  Arroyo  Genter  evaluated  AWALP  (using  data 
from  104  students  enrolled  in  three  AWALP  courses  in  2013), 
addressing  multiple  aspects  of  individual  and  team  adaptive 
performance  and  identifying  potential  areas  for  improvement 
in  AWALP  curriculum  and  delivery.  The  RAND  Arroyo  Gen¬ 
ter  team  also  provided  a  set  of  instruments  and  protocols  to 
foster  ongoing  assessment  and  improvement  in  AWALP  and 
in  other  courses  or  events  that  include  adaptability  training. 

How  Effective  Is  AWALP? 

The  evaluation  used  multiple  measures  of  adaptive  performance — 
including  piloting  new  ones — to  assess  a  range  of  training 
outcomes.  The  evaluation  assessed  reactions  to  the  course — 


Key  findings: 

•  Students  were  extremely  satisfied  with  the  course  struc¬ 
ture,  content,  and  delivery,  attributing  learning  largely  to 
course  content  (adaptability)  and  the  training  methods. 

•  There  was  substantial  improvement  in  self-efficacy  and 
interest  and  an  increase  in  the  perceived  need  for  adapt¬ 
ability  and  related  leader  behaviors  in  the  students'  cur¬ 
rent  jobs. 

•  Students  showed  a  substantial  increase  in  knowledge  of 
AWALP  concepts. 

•  AWALP  graduates  reported  substantial  application  of 
adaptive  performance  principles  on  the  job,  especially 
in  coaching,  training,  delegating  to  subordinates,  and 
seeking  subordinate  input. 

•  AWALP,  supported  by  systematic  course  evaluation, 
provides  a  promising  approach  for  training  as  the  Army 
seeks  to  further  develop  adaptable  leaders  and  teams. 


satisfaction  with  course  content,  design,  and  delivery.  It  also 
assessed  changes  in  attitudes  and  learning  using  pretraining 
and  posttraining  surveys  and  tests,  respectively.  Researchers 
also  developed  a  measure  to  assess  adaptability  at  the  team 
level  for  practical  exercises,  using  both  the  students  and  guides 
as  raters  of  team  adaptive  performance.  The  measure  required 
raters  to  assess  whether  the  dimensions  of  adaptive  performance 
were  required  in  each  of  three  exercises  and  to  rate  the  team’s 
performance  on  the  dimensions.  Finally,  the  evaluation  assessed 
perceived  transfer  performance — how  training  results  in  payoffs 
to  the  organization — by  conducting  telephone  interviews  with 
AWALP  graduates  and  their  supervisors  to  assess  how  graduates 
apply  AWALP  principles  on  the  job  and  the  longer-term  impact 
of  the  course  on  adaptability  behavior  and  attitudes. 

Overall,  the  results  of  this  evaluation  provide  evidence  of 
AWALP  success  across  a  range  of  measures.  Table  1  highlights 
key  findings  by  outcomes. 
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Table  1.  Key  Findings  from  AWALP  Evaluation 


Outcomes 

Key  Findings 

Reactions  to  AWALP 

•  Students  were  extremely  satisfied  with  the  course  structure,  content,  and  deiivery,  attributing 
learning  largely  to  course  content  (adaptive  performance)  and  the  training  methods,  which  differ 
substantiaily  from  typical  Army  training.  All  students  reported  that  they  would  recommend  the 
course  to  others. 

•  30  percent  of  students  anticipated  that  leader  buy-in  and  20  percent  anticipated  that  command 
climate  would  pose  challenges  to  applying  adaptability  concepts  on  the  job. 

Attitudes  Toward  Adaptability 

•  There  was  substantial  improvement  in  seif-efficacy  for,  and  interest  in,  being  adaptable,  even  after 
accounting  for  students'  individual  characteristics  associated  with  adaptive  performance. 

•  Results  suggest  an  increase  in  the  perceived  need  for  adaptive  performance  in  students'  current  jobs. 
Thus,  students  not  oniy  sensed  a  change  in  their  own  capabilities  and  interests  about  adaptability  but 
appeared  to  see  their  work  contexts  in  a  different  light. 

Knowledge  About  Course  Concepts 

•  Students  showed  increased  knowledge  of  AWALP  concepts,  with  average  scores  on  a  multiple-choice 
test  of  60  percent  correct  pretraining  and  76  percent  correct  posttraining. 

•  AWALP  was  successful  at  fostering  knowledge  gain  for  a  wide  range  of  students. 

Team  Adaptability 

•  Students'  and  guides'  ratings  of  requirements  for  adaptive  performance  generally  were  similar 
to  each  other  across  the  three  exercises.  Evidence  for  improved  student  accuracy  in  ratings  of 
requirements  over  time  was  mixed. 

•  Both  students  and  guides  gave  favorable  ratings  of  team  performance,  but  students  gave  higher 
ratings  and  seemed  to  become  more  confident  about  the  level  of  team  adaptability  as  the  course 
progressed. 

Perceived  Transfer  of  Training 

•  Graduates  reported  substantial  application  of  AWALP  principles,  especially  in  coaching,  training, 
delegating  to  subordinates,  and  seeking  subordinate  input. 

•  While  AWALP  principles  were  successfully  disseminated  to  graduates'  subordinates,  dissemination  was 
somewhat  less  successful  to  peers,  commanders,  or  throughout  the  unit. 

•  Graduates  remained  positive  about  AWALP  training  but  found  command  climate  and  entrenched 
leadership  the  biggest  obstacles  to  applying  AWALP  principles. 

•  Graduates'  supervisors  were  positive  about  the  effect  of  AWALP  on  graduates,  but  interpreting  results 
requires  caution  because  the  sample  size  was  small. 

What  Are  Some  Recommendations  and  Lessons 
Going  Forward? 

Students  had  few  recommended  improvements  to  AWALP, 
but  other  evaluation  results  point  to  ways  to  improve  AWALP 
and  the  future  evaluation  of  the  course.  Table  2  summarizes 
key  recommendations. 

For  related  efforts  by  the  U.S.  Army’s  Training  and 
Doctrine  Command  (TRADOC)  to  train  adaptability, 
the  RAND  Arroyo  Center  team  proposed  two  options 
for  expanding  the  AWALP  approach.  One  option  is  to 
increase  the  number  of  students  receiving  training,  either 
by  increasing  throughput  in  the  current  course  or  by  con¬ 
tinuing  AWG’s  efforts  to  stand  up  local  versions  of  AWALP 
by  training  local  trainers  through  mobile  training  teams. 
Another  way  to  disseminate  AWALP  more  broadly  is  to 
incorporate  adaptability  principles  into  existing  professional 
military  education  courses,  such  as  advanced  leader  courses 


for  noncommissioned  officers.  And  while  AWALP  provides  a 
starting  point  for  training  soldiers  to  work  in  and  lead  teams, 
TRADOC  can  support  mission  command  principles  further 
by  creating  a  follow-on  course  that  expands  instruction  at  the 
team  level,  addressing  such  topics  as  shared  mental  models, 
transactive  memory  systems,  team  trust,  and  team  facilitation. 

Conclusion 

The  shift  in  Army  doctrine  from  command  and  control  to 
mission  command  calls  for  profound  changes  in  leader  and 
team  conduct,  with  a  concomitant  transformation  in  train¬ 
ing.  AWG’s  successful  development  and  implementation  of 
AWALP  exemplifies  mission  command  principles  in  terms  of 
both  the  content  of  the  course  and  how  it  is  taught.  AWALP, 
supported  by  systematic  course  evaluation,  provides  a  prom¬ 
ising  approach  for  the  Army  as  it  seeks  to  further  develop 
adaptable  leaders  and  teams. 
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Table  2.  Key  Recommendations 


Areas 

Key  Recommendations 

Improvements  to  AWALP 

•  Put  more  emphasis  in  the  curriculum  on  responding  to  potential  challenges  in  implementing  adaptive 
performance  principles  on  the  job. 

•  Provide  additional  feedback  to  students  about  individual  and  team  adaptive  performance. 

•  Integrate  additional  concepts  about  team  adaptability  into  the  curriculum. 

Ongoing  Evaluation  of  AWALP 

•  Continue  to  administer  the  knowledge  test,  questions  about  attitudes  toward  adaptability  and  related 
leader  behaviors,  reactions  to  the  course,  and  ratings  of  teams  but  make  some  changes  to  reduce  the 
burden  on  respondents. 

•  Assess  additional  aspects  of  team  performance  associated  with  adaptive  performance. 

•  Continue  to  assess  training  transfer  through  interviews  of  course  graduates  to  assess  how  they  are 
using  AWALP  principles  and  to  identify  lessons  learned  to  improve  the  course. 

Future  Evaluation  of  AWALP 

•  Conduct  systematic  behavioral  observations  during  the  course  to  further  evaluate  individual  and  team 
adaptive  performance. 

•  Assess  transfer  of  training  by  measuring  the  association  of  individual  performance  in  training  with 
subsequent  job  performance,  where  graduates'  supervisors  would  provide  independent,  quantifiable 
ratings  of  job  performance. 

•  Modify  recruiting  processes  for  supervisors  to  improve  success  in  obtaining  their  feedback. 
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